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Abstract 

We  propose  algorithms  for  the  design  of  FIR  com¬ 
paction  filters,  which  find  applications  in  FIR  subband 
coders.  The  techniques  produce  compaction  gains  very 
close  to  that  of  optimal  compaction  filters,  for  any 
fixed  filter  order  and  input  autocorrelation.  The  main 
theme  of  the  paper  is  the  design  of  multistage  FIR  com¬ 
paction  filters  based  on  an  iterated  linear  programming 
approach.  The  theory  behind  this  is  presented  followed 
by  design  examples  and  comparisons.  Also,  a  noniter¬ 
ative  algorithm  much  faster  than  other  iterative  opti¬ 
mization  techniques  (e.g.  linear  programming)  will  be 
briefly  mentioned.  Further  details  of  noniterative  tech¬ 
niques  will  be  presented  elsewhere. 


filter  orders  are  high,  the  computational  complexity  of 
the  hnear  programming  technique  becomes  very  high. 
We  propose  an  algorithm  to  design  the  filters  in  mul¬ 
tiple  stages  resulting  in  efficiency  in  both  design  and 
implementation  phases.  We  also  briefly  describe  a  non¬ 
iterative  design  technique  called  the  window  method. 
Also  mentioned  is  a  new  analytical  method  in  the  two- 
channel  case.  The  details  of  these  noniterative  design 
methods  will  be  presented  elsewhere  [4]. 

2  The  FIR  energy  compaction  problem 

Let  H{z)  be  an  FIR  filter  of  order  N.  Consider 
Fig,  1  where  the  input  x(n)  is  a  zero-mean  WSS  random 
process  with  the  power  spectreJ  density  The 


1  Introduction 

We  will  describe  some  efficient  methods  to  design  FIR 
compaction  filters.  These  filters  find  application  in 
M—channel  FIR  orthonormal  filter  banks  [11].  Because 
of  this  basic  application,  we  will  refer  to  “M— channel 
compaction  filters” ,  although  there  will  be  only  one  fil¬ 
ter  to  work  with.  The  theory  and  design  details  depend 
on  M. 

Compaction  filters  have  attracted  a  great  deal  of 
attention  due  mainly  to  the  recently  discovered  fact 
[9,  13]  that  they  are  the  building  blocks  of  optimal  or¬ 
thonormal  (paraunitary)  filter  banks.  This  connection 
however  is  made  for  the  case  where  the  filters  are  al¬ 
lowed  to  be  ideal.  A  number  of  authors  considered  the 
finite  order  (FIR)  compaction  filter  design  problem  for 
the  tw^o-channel  case  [2,  1,  14,  10,  6,  8]  and  for  the 
M— channel  case  [7]. 

It  is  possible  to  design  optimal  FIR  compaction  fil¬ 
ters  by  using  linear  programming.  However,  when  the 


x(n)  H{z)  [— y(«) 


Figure  1:  The  FIR  energy  compaction  filter. 


output  of  the  filter  is  decimated  by  M  to  produce  y(n). 
The  optimum  FIR  energy  compaction  problem  is  to 
maximize  the  variance 

al  =  r  (1) 

J  — 


of  y{n)  subject  to  the  Nyqi!8st(M)  condition  [11]  on 
G{e^^)  =  Let  the  impulse  response  of 

G{e^^)  be  gin).  Then,  the  Nyquist(M)  condition  is 
giMn)  =  S{n).  Notice  that  by  definition  Gie^^)  >  0. 
Define  the  compaction  gain  as 


Gcomp(M,N)  =  ^  = 


(2) 
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where  is  the  variance  of  x(n).  The  aim  therefore  is 
to  maximize  the  compaction  gain. 
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As  described  in  [4],  the  case  where  N  <  M  and 
the  case  where  ideal  filters  are  allowed  are  solved  an¬ 
alytically.  Our  interest  is  therefore  for  the  case  where 
M  <N  <00. 

3  Linear  Programming 

The  use  of  linear  programming  method  in  compaction 
filter  design  was  proposed  in  [6],  and  is  reviewed  next. 
Assume  that  the  input  process  x(n)  is  real  with  the 
autocorrelation  sequence  r(n).  The  output  variance 
can  be  written  £is 

N 

=  »-(0)  +  2  52  9in)r{n)  (3) 

n=l 

Let  gj  and  r<j  be  the  vectors  formed  by  the  nonzero 
components  of  g{n)  and  r(n)  for  n  =  1, . . .  A^.  That  is, 

gd  =  [ff(l)  5(2)  . . .  s(M  -  1)  s(M  -Hi)  . . .  giN)f, 
Vd  =  [r(l)  r(2)  . . .  r(M  -  1)  r(M  -I- 1)  . . .  t{N)Y . 

Then  (3)  can  be  written  as  cr^  =  r(0)  -h  2rjgd. 
This  incorporates  the  Nyquist(M)  condition  but 
not  the  nonnegativity  constraint  on  Let 

c^(a;)  =  [cos(a;)  cos(2a;)  . . .  cos((M  -  1)cj)  cos((M  + 
l)u;)  ...  cos(A^a;)]^.  Then  G{e^^)  =  1 -f  2cJ(a;)gd. 
Hence  the  problem  is  equivalent  to  the  following: 

maximize  rjgd, 

subject  to  cj(a;)gd  >  —0.5,  Vcj  6  [0,7r]. 

This  type  of  problem  is  typically  classified  as  semiin¬ 
finite  linear  programming  [6].  By  discretizing  the  fre¬ 
quency,  one  reduces  this  to  a  well  known  standard  lin¬ 
ear  programming  problem. 

Example  1.  Let  the  input  process  be  AR(5)  as  in 
page  37  of  [3]  which  is  used  to  model  speech  signals. 
Let  M  =  8  and  iV  =  15  and  let  us  use  L  =  32  uniform 
frequencies,  uJk  =  k27r/L^  A  =  0, . . . ,  L  —  1.  We  obtain 
the  frequency  response  G{e^^)  shown  in  Fig.  2  which  is 
not  nonnegative  for  all  frequencies.  In  order  to  obtain 
a  compaction  filter,  we  must  have  G{e^^)  >  0.  One 
way  to  guarantee  this  is  to  “lift”  G{e^^)  by  increasing 
^(0)  relative  to  the  other  coefficients  (since  ^(0)  has  to 
be  1,  in  effect  we  rescale  g{n)  for  n  ^  0  by  a  constant 
that  is  slightly  less  than  1).  For  this  example,  it  turns 
out  that  this  constant  is  about  0.8044.  The  resulting 
comp^tion  gain  is  3.7873.  Optimum  ideal  compaction 
filter  gives  a  gain  of  4.9721. 

In  the  next  section  we  propose  a  new  technique  to 
overcome  the  difficulty  without  having  to  find  the  min¬ 
imum  of  Gie^"^)  whidi  is  necessary  to  decide  on  how 
much  to  lift  it. 


3.1  Windowing  of  the  linear  programming 
solution 

Consider  the  periodical  expansion  of  the  linear 

programming  solution  where  L  is  the  number  of  dis¬ 
crete  uniform  frequencies  used  in  the  design  process. 
Assume  that  L  >  2iV.  Linear  programming  assures 
that  Gie^"^)  is  nonnegative  at  the  uniform  frequencies. 
Hence  the  Fourier  series  coefficients  Giik)  of  gi(n)  $xe 
nonnegative.  Now  consider  the  product 

w{n)gL{n)  (4) 

where  w{n)  is  a  symmetric  window  of  length  2K  4-  1 
(where  K  <  L  -  N,  see  Fig.  3).  If  w{n)  has  nonnega¬ 
tive  Fourier  transform  W(e*^'“'),  the  Fourier  transform  of 
the  product  is  nonnegative  as  well.  The  reason  follows 
from  the  fact  that  the  Fourier  transform  of  w{n)gL  {n) 
is  a  weighted  sum  of  shifted  versions  of  W(e^“'),  with 
nonnegative  weights.  For  maximum  compaction  gain, 
the  symmetric  order  of  w{n)  is  chosen  to  be  maximum, 
namely  K  L  -  N  -  1.  One  can  use  a  fixed  window 


Figure  3;  Windowing  of  the  linear  programming 
solution. 

like  a  triangular  window  as  depicted  in  the  figure  and 
get  a  satisfactory  compaction  gain.  However  one  can 


always  optimize  the  window.  The  optimum  w{n)  is  the 
autocorrelation  sequence  of  the  maximal  eigenfilter  of 
the  K  X  K  Hermitian  Toeplitz  matrix  formed  by  the 
product  r{n)gL(n)  [4],  Since  the  window  length  2K-h  1 
is  very  high  in  linear  programming  designs,  we  suggest 
to  use  a  triangular  window  rather  than  optimizing  the 
window.  The  performance  loss  is  negligibly  small. 
Example  2.  Consider  the  previous  example.  Using  a 
triangular  window  of  order  iC  =  L  —  —  1  =  16,  we 

have  the  resulting  compaction  gain  of  4.4967.  This  is 
significantly  better  than  that  of  the  previous  “lifting” 
technique  ir-  Example  1.  When  we  further  optimize  the 
window,  we  find  that  the  compaction  gain  is  4.6769.  If 
L  —  256  however,  we  have  the  scaling  constant  0.9986 
in  the  lifting  method  with  a  corresponding  compaction 
gain  of  4.8471.  If  we  use  triangular  window  of  order 
L  —  N  —  1  =  240,  we  have  the  compaction  of  4.8236 
£ind  if  we  optimize  the  window  we  have  the  compaction 
gain  of  4.8539.  Recall  that  the  ideal  compaction  gain 
was  4.9721. 

Example  3.  Let  the  input  be  psd  be  as  in  Fig.  4  and 
let  AT  =  65  and  M  =  2.  In  the  same  figure,  we  plot  the 
magnitude  square  of  the  compaction  filter 

H{z)  designed  by  the  linear  programming  method. 


Figure  4:  The  psd  of  an  AR(5)  process,  and 
the  magnitude  square  of  an  optimal  compaction 
filter  designed  by  linear  programming  (N  =  65, 
M  =  2). 

The  number  of  frequencies  used  in  the  design  pro¬ 
cess  was  L  =  512.  We  have  used  triangular  window  of 
symmetric  order  K  —  L  N  --1  —  446  and  found  that 
the  resulting  compaction  gain  is  1.8698.  If  we  optimize 
the  window  the  compaction  gain  becomes  1.8744.  If 
we  “lift”  then  we  have  the  compaction  gain  of 

1.8713.  One  can  verify  that  the  compaction  gain  of  the 
ideal  (infinite  order)  compaction  filter  is  1.8754. 

4  Multistage  FIR  (IFIR)  compaction 
filter  design 

Let  M  =  MqMi  and  consider  Fig.  5(a).  This  can 
be  redrawn  as  in  Fig.  5(b).  The  equivalent  filter  is 


H{z)  =  Ho{z)Hi{z^^).  We  will  first  impose  the 
Nyquist(M)  condition  only  on  |i7(e^“^)l^.  Later  we 
will  impose  Nyquist  conditions  on  individual  filters  that 
guarantee  the  Nyquist(M)  property  of  We 

will  describe  the  details  of  how  to  find  Hi  (z)  for  a  fixed 
Ho{z)  and  vice  versa,  in  an  iterative  maimer. 


Figure  5:  Multistage  compaction  filter  design, 
(a)  Basic  configuration,  (b)  Equivalent  system. 

Let  Goie^'^)  =  \Ho{e^^)\\  Gi(e>“)  = 
and  G{e^^)  =  with  impulse  responses 

^o(n),  gi(n),  and  g(n)  respectively.  Denote  the  orders 
of  Ho{z),  Hi{z),  and  H{z)  by  Nq,  Ni,  and  N  respec¬ 
tively.  Hence  we  have  N  =  MqNx  -h  A^o-  Define 
go  =  [ffo(O)  flo(l)  ...  ffo(iVo)F, 
gi  =  [<7i(0)?i(l)  ...  SiW)F, 
g  =  [g{0)g{l)  ...  g{N)f- 

Optimization  of  Hi{z)  for  a  given  Ho{z).  We  have 
G{z)  =  Go{z)Gi{z^^).  Let  Go  be  the  {2N  +  1)  x 
(2MoA/'i  -f  1)  convolution  matrix  formed  by  ^o(^)*  Tak¬ 
ing  into  account  the  sjnnmetries  and  the  fact  that 
Gi(z^°)  has  nonzero  components  only  for  multiples 
of  Mo,  we  can  write  g  =  Aogi,  where  Aq  is  an 
(N  +  1)  X  (A^i  -f  1)  matrix  that  is  obtained  from  Go- 
Now,  the  Nyquist(M)  constraint  requires  that  if  we 
decimate  g  by  M  we  should  get  e©  =  [1  0  ...  0]^.  Let 
Bo  denote  the  matrix  that  is  obtained  by  taking  every 
Mth  row  of  Aq.  Then  we  should  have  Bogi  =  gq. 
To  force  the  nonnegativity  constraint  on  Gi(e-^‘^),  let 

Co  (a;)  =  [1  2cos(a;)  2cos(2u;)  ...  2cos(Aricj)]^.  Then 
the  constraint  ^1(6*^“^)  >  0  becomes  co^(a;)gi  > 
0,  Vo;  G  [0,7r].  If  r  =  [r(0)  2r(l)  ...  2r(7V)]^,  the 
objective  is  to  maximize  r^g  =  r^Aogi.  Hence  we 
have  reduced  the  problem  to  the  following: 

maximize  ro^gi, 

subject  to  Bogi  =  eo,  and  co^(u^)gi  >0,  Vo;  G  [0,7r], 

where  ro  =  Ao^r.  Hence  a  standard  linear  program¬ 
ming  algorithm  can  be  applied,  once  a  set  of  frequencies 
is  chosen  for  the  inequality  constraint. 

Optimization  of  Ho{z)  for  a  given  Hi{z),  Similarly, 
one  can  reduce  the  problem  of  finding  the  best  Ho{z) 
for  a  given  Hi{z)  to  the  following  linear  programming 


problem: 

maximize  ri^go, 

subject  to  Bigo  =  eo,  and  ci^(a;)go  >  0,  Vo;  E  [0,7r], 

where  ci^)  =  [1  2cos(a;)  2cos(2u;)  ...  2cos(iVba;)]^, 
n  =  Ai  r.  The  (iV  +  1)  x  {Nq  -h  1)  matrix  Ai  is 
obtained  from  the  {2N  +  1)  x  {2No  4- 1)  convolution 
matrix  formed  by  gi{n)  by  taking  the  symmetries  into 
account  and  the  matrix  Bi  is  obtained  by  taking  every 
Afth  row  of  Ai. 

One  can  iterate  between  the  above  two  optimiza¬ 
tion  steps  until  there  is  no  significant  change  in  the 
compaction  gain.  The  initial  choice  of  5o(^)  can  sigmf- 
icantly  affect  the  resulting  compaction  gain.  According 
to  our  design  experience  if  yo(^)  is  chosen  to  be  a  trian¬ 
gular  sequence,  the  compaction  gain  at  the  end  of  the 
iteration  is  very  good.  The  filters  ^o(^)  gi  (n)  which 
result  from  the  iteration  should  spectrally  be  factorized 
to  identify  Ho{z)  and  Hi{z).  This  step  will  be  success¬ 
ful  only  if  the  solutions  are  such  that  Go{e^^)  >  0  and 
>  0  for  all  cj.  If  this  is  not  the  case,  we  can 
force  it  by  use  of  windowing  on  5o(^)  and  gi{n)  as  de¬ 
scribed  in  Sec.  3.1  or  by  the  “lifting”  technique.  If 
this  is  done  then  the  product  filter  Go{z)Gi{z^°)  will 
not  be  exactly  Nyquist(M).  In  the  next  subsection  we 
show  how  to  overcome  this  problem. 

Example  4.  Let  us  design  IFIR  compaction  filters 
for  the  pair  (M,  AT)  =  (36,65),  and  for  the  input  pro¬ 
cess  whose  psd  wats  given  in  Fig.  4.  Let  Mq  =  9  and 
Ml  =  4,  and  let  Nq  =  11  so  that  Ny  =  6.  The  number 
of  frequencies  used  in  the  designs  is  L  =  1024.  Start¬ 
ing  with  a  triangular  sequence  for  ^o(^)>  algorithm 
converges  in  a  few  steps.  We  windowed  the  resulting 
solutions  goifi)  and  gi{Ti)  with  triangular  windows  of 
symmetric  orders  L— iVo*— 1  and  L— iVi  — 1  respectively. 
The  final  product  filter  was  not  exactly  Nyquist(M) 
because  it  weis  found  that  ^(36)  cf  —0.0018  /  0.  The 
final  competition  gain  was  5.1444.  If  we  design  a  com¬ 
paction  filter  of  order  18  directly  (i.e.,  not  using  IFIR 
technique),  the  compaction  gain  is  4,4225.  This  cor¬ 
responds  to  a  compaction  filter  with  the  same  number 
of  active  multipliers,  namely  19.  If  we  design  a  com¬ 
paction  filter  of  order  65  directly  (66  active  multipliers), 
then  the  resulting  compaction  gain  is  7.2337. 

4.1  A  Particular  IFIR  configuration 

In  Fig.  5,  if  Go{z)  is  Nyquist(Mo)  and  Gi{z)  is 
Nyquist(Mi),  it  can  be  verified  that  G{z)  given  by 
Go{z)Gi{z^^)  is  Nyquist(M).  Now,  let  us  fix  Ho{z) 
to  be  a  ^id  compaction  filter  for  the  pair  {Nq,Mo). 
Referring  to  Fig.  6(a),  the  best  Hi{z)  is  the  optimum 
compaction  filter  for  My),  and  for  the  input  xo{n) 


which  has  the  psd 

Similarly,  if  Hi  (2)  is  a  fixed  compaction  filter  for  the 


x(n). 


x(n)  ■ 


-jKn) 


(b) 


Figure  6:  Special  IFIR  design  configuration. 

pair  (Hi,  Ml),  then  we  can  redraw  the  configuration  as 
in  Fig.  6(b).  The  best  Ho(z)  is  the  optimum  com¬ 
paction  filter  for  (Ho.Mo),  and  for  the  input  xi(n) 
which  has  the  psd  5ji*i(z)  =  Gi{z^°)Sxx{z)-  0*^® 
can  design  the  compaction  filters  Ho{z)  and  Hi{z)  it¬ 
eratively  using  any  of  the  known  techniques.  Hence, 
one  can  use  the  linear  programming  technique  as  well 
as  any  other  technique  like  the  noniterative  methods  to 
be  mentioned  in  the  next  section.  Also  note  that  if  the 
ideal  filters  are  allowed,  this  multistage  configuration 
has  no  loss  of  generality  as  shown  in  [5]. 

Example  5.  Let  the  setup  be  the  same  as  in  the  previ¬ 
ous  example.  We  have  designed  the  compaction  filters 
Ho (2)  and  Hi{z)  iteratively  using  the  standard  linear 
programming  procedure  as  in  Example  3.  We  have 
started  with  Hi(z)  =  1.  The  first  compaction  filter 
Ho  (2)  is  therefore  the  optimal  compaction  filter  for  the 
pair  {Mo,  No)  =  (9, 11)  for  the  original  autocorrelation 
sequence.  We  have  windowed  the  final  product  filters 
as  we  did  in  Example  3  to  guarantee  the  nonnegativity. 
The  resulting  overall  compeiction  g^dn  is  4.9432.  This  is 
slightly  smaller  than  the  overall  compaction  gain  5.1444 
in  Example  3.  However,  the  resulting  overall  filter  here 
is  exactly  Nyquist(M)  unlike  the  case  of  Example  4. 

4.2  Noniterative  techniques 


In  [4]  we  propose  two  noniterative  methods  for  the 
design  of  FIR  compaction  filters.  One  is  for  any 
number  of  channels  while  the  other  is  for  the  special 
two-channel  case.  The  first  one  is  called  the  win¬ 
dow  method.  Although  it  is  suboptimal,  the  window 
method  is  applicable  for  any  process  including  complex 
ones.  It  has  finite  number  of  elementary  steps  and  the 
resulting  compaction  gains  are  very  close  to  the  opti¬ 
mal  ones  especially  for  high  filter  orders.  The  second 
method  is  called  the  analytical  method.  It  finds  the 
optimal  solution,  but  it  is  applicable  for  a  restricted 
class  of  random  processes.  Because  of  its  relevance  and 
similarities  to  the  linear  programming  method,  we  will 


describe  the  window  method  briefly.  The  details  of 
both  methods  are  presented  in  [4]. 

Window  method.  The  main  idea  is  to  write  the  im¬ 
pulse  response  of  =  |H(e^")|2  in  the  form: 

gin)  =  fft(n)ti)(n)  (5) 

where  giin)  is  periodic(L),  and  ti;(n)  has  nonnegative 
Pourier  transform.  The  method  takes  the  window  to 
be  a  triangular  one,  and  finds  the  optimum  giin).  By 
fixing  this  giin),  the  window  w(n)  is  then  optimized. 
Optimizing  giin)  for  a  fixed  win)  is  done  by  compar¬ 
ing  the  psd  at  M  alias  frequencies  for  eadi  Wk  =  k2ir/ L 
and  assigning  values  to  Giik)  accordingly.  Here  Gt(fc) 
is  the  Fourier  series  coefficients  of  giin)-  This  can  be 
considered  as  finite  version  of  the  algorithm  in  [12]. 
Optimizing  win)  for  a  fixed  giin)  is  an  extremal  eigen¬ 
vector  problem  [4]. 

Relation  to  linear  programming.  In  linear  pro¬ 
gramming  we  find  a  sequence  whose  Fourier  transform 
is  nonnegative  at  some  prescribed  set  of  L  frequen¬ 
cies.  If  these  are  chosen  to  be  L  imiform  frequencies 
Wk  =  k2-nlL,  fc  =  0, . . . ,  L  -  1,  then  we  can  associate 
a  periodic  sequence  giin)  whose  Fourier  series  coeffi¬ 
cients  are  nonnegative.  If  we  window  this  with  win) 
whose  Fourier  transform  is  nonnegative  as  in  Sec.  3.1, 
we  guarantee  the  nonnegativity  of  Gie?'^).  Hence  in 
principle,  we  have  the  same  form  for  gin)  as  in  (5). 
There  are  some  basic  differences  however:  In  linear 
programming  gbin)  is  automatically  restricted  to  be  of 
finite  length.  That  is,  it  is  guaranteed  that  giin)  =  0 
for  AT  <  |n|  <  L- A'’.  This  is,  in  general,  not  true  in  the 
window  method.  The  order  of  win)  should  be  the  same 
as  that  of  gin)  in  the  window  method  while  this  is  not 
necessary  in  the  linear  programming  method.  In  the 
special  case  where  L  =  2N ,  the  two  methods  become 
the  same!  Hence  the  window  method  becomes  an  effi¬ 
cient  way  of  solving  a  linear  programming  problem.  In 

[4]  we  give  a  detailed  comparison  of  the  two  methods. 

5  Remarks  and  Conclusions 

We  have  discussed  some  efficient  design  methods  for 
FIR  compaction  filters.  We  first  proposed  a  simple 
way  to  guarantee  the  noimegativity  of  the  linear  pro¬ 
gramming  solutions.  Then  we  have  considered  multi¬ 
stage  extensions.  These  offer  reduction  in  both  design 
and  implementation  complexity.  We  have  also  briefly 
described  a  noniterative  technique  called  the  window 
method  and  discussed  its  relevance  to  the  linear  pro¬ 
gramming  technique.  When  the  number  of  frequencies 
in  the  linear  programming  is  moderate,  we  have  seen 
that  the  “lifting”  technique  to  assure  nonnegativity  of 
Gie^'^)  resulted  in  significant  loss.  In  these  cases  op¬ 


timization  of  the  window  in  the  windowing  technique 
was  the  best.  When  the  number  of  frequencies  is  high 
however,  then  either  “lifting”  or  the  use  of  a  triangular 
window  resulted  in  very  little  loss  in  compaction  gain. 
Finally  we  noted  that  when  L  =  2N,  the  linear  pro¬ 
gramming  and  the  window  methods  become  the  same. 
Since  the  window  method  is  much  faster,  it  should  be 
preferred  to  the  linear  programming  technique  if  L  is 
chosen  to  be  moderate  (e.g.,  not  much  larger  than  2N). 
We  conclude  the  paper  by  referring  to  an  important  ob¬ 
servation  made  in  [4]  that  states  that  if  the  filter  order  is 
relatively  high,  then  the  line^  programming  technique 
can  be  avoided  altogether  because  the  choice  L  =  2N 
yields  very  good  window-based  design. 
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